Harddisk Images based on redologs
This section describes how the three new disk images "undoable", "growing", and "volatile" are implemented in Bochs 2.1 :
undoable -> flat file, plus growing, commitable, rollbackable redolog file
growing -> growing files, all previously unwritten sectors go to the end of file
volatile -> flat file, plus hidden growing redolog
Description
The idea behind volatile and undoable disk images is to have a flat file, associated with one redolog file.
Reading a sector is done from the redolog file if it contains the sector, or from the flat file otherwise.
Sectors written go to the redolog, so flat files are opened in read only mode in this configuration.
The redolog is designed in a way so it starts as a small file and grows with every new sectors written to it. Previously written sectors are done in place. Redolog files can not shrink.
The redolog is a growing file that can be created on the fly.
Now, it turns out that if you only use a redolog without any flat file, you get a "growing" disk image.
So "undoable", "volatile" and "growing" harddisk images classes are implemented on top of a redolog class.
How redologs works ?
At the start of a redolog file, there is a header, so Bochs can check whether a file is consistent. This header could also be checked when we implement automatic type and size detection.
The generic part of the header contains values like type of image, and spec version number.
The header also has a specific part. For redologs, the number of entries of the catalog, the extent, bitmap and disk size are stored.
In a redolog, the disk image is divided in a number of equal size "extents". Each extent is a collection of successive 512-bytes sectors of the disk image, preceeded by a n*512bytes bitmap.
the n*512bytes bitmap defines the presence (data has been written to it) of a specific sector in the extent, one bit for each sector. Therefore with a 512bytes bitmap, each extent can hold up to 4k blocks
Typically the catalog can have 256k entries. With a 256k entries catalog and 512bytes bitmaps, the redolog can hold up to 512GiB
Note: All data is stored on images as little-endian values
Header
At the start of a redolog file, there is a header. This header is designed to be reusable by other disk image types.
The header length is 512 bytes. It contains :
Table 2-4. Generic header description
| Start position in bytes | Length in bytes | Data type | Description | Possible values |
|---|---|---|---|---|
| 0 | 32 | string | magical value | Bochs Virtual HD Image |
| 32 | 16 | string | type of file | Redolog |
| 48 | 16 | string | subtype of file | Undoable, Volatile, Growing |
| 64 | 4 | Bit32u | version of used specification | 0x00010000 |
| 68 | 4 | Bit32u | header size | 512 |
Parameters
The following tables shows what paremeters are used when creating redologs or creating Θgrowing" images :
Table 2-6. How number of entries in the catalog and number of blocks by extents are computed
| Catalog entries | Catalog size(KiB) | Bitmap size (B) | Extent size (KiB) | Disk Max Size |
|---|---|---|---|---|
| 512 | 2 | 1 | 4 | 2MiB |
| 512 | 2 | 2 | 8 | 4MiB |
| 1k | 4 | 2 | 8 | 8MiB |
| 1k | 4 | 4 | 16 | 16MiB |
| 2k | 8 | 4 | 16 | 32MiB |
| 2k | 8 | 8 | 32 | 64MiB |
| 4k | 16 | 8 | 32 | 128MiB |
| 4k | 16 | 16 | 64 | 256MiB |
| 8k | 32 | 16 | 64 | 512MiB |
| 8k | 32 | 32 | 128 | 1GiB |
| 16k | 64 | 32 | 128 | 2GiB |
| 16k | 64 | 64 | 256 | 4GiB |
| 32k | 128 | 64 | 256 | 8GiB |
| 32k | 128 | 128 | 512 | 16GiB |
| 64k | 256 | 128 | 512 | 32GiB |
| 64k | 256 | 256 | 1024 | 64GiB |
| 128k | 512 | 256 | 1024 | 128GiB |
| 128k | 512 | 512 | 2048 | 256GiB |
| 256k | 1024 | 512 | 2048 | 512GiB |
| 256k | 1024 | 1024 | 4096 | 1TiB |
| 512k | 2048 | 1024 | 4096 | 2TiB |
| 512k | 2048 | 2048 | 8192 | 4TiB |
| 1024k | 4096 | 2048 | 8192 | 8TiB |
| 1024k | 4096 | 4096 | 16384 | 16TiB |
| 2048k | 8192 | 4096 | 16384 | 32TiB |
Redolog class description
The class redolog_t(); implements the necessary methods to create, open, close, read and write data to a redolog. Managment of header catalog and sector bitmaps is done internally by the class.
Constants
#define STANDARD_HEADER_MAGIC "Bochs Virtual HD Image" #define STANDARD_HEADER_VERSION (0x00010000) #define STANDARD_HEADER_SIZE (512) |
#define REDOLOG_TYPE "Redolog" #define REDOLOG_SUBTYPE_UNDOABLE "Undoable" #define REDOLOG_SUBTYPE_VOLATILE "Volatile" #define REDOLOG_SUBTYPE_GROWING "Growing" |
#define REDOLOG_PAGE_NOT_ALLOCATED (0xffffffff) |
Methods
redolog_t(); instanciates a new redolog.
int make_header (const char* type, Bit64u size); creates a header structure in memory, and sets its type and parameters based on the disk image size. Returns 0.
int create (const char* filename, const char* type, Bit64u size); creates a new empty redolog file, with header and catalog, named filename of type type for a size bytes image. Returns 0 for OK or -1 if a problem occured.
int create (int filedes, const char* type, Bit64u size); creates a new empty redolog file, with header and catalog, in a previously opened file described by filedes, of type type for a size bytes image. Returns 0 for OK or -1 if a problem occured.
int open (const char* filename, const char* type, Bit64u size); opens a redolog file named filename, and checks for consistency of header values against a type and size. Returns 0 for OK or -1 if a problem occured.
void close (); closes a redolog file.
off_t lseek (off_t offset, int whence); seeks at logical data offset offset in a redolog. offset must be a multiple of 512. Only SEEK_SET is supported for whence. Returns -1 if a problem occured, or the current logical offset in the redolog.
ssize_t read (void* buf, size_t count); reads count bytes of data of the redolog, from current logical offset, and copies it into buf. count must be 512. Returns the number of bytes read, that can be 0 if the data has not previously be written to the redolog.
ssize_t write (const void* buf, size_t count); writes count bytes of data from buf to the redolog, at current logical offset. count must be 512. Returns the number of bytes written.
Disk image classes description
"volatile" and "undoable" disk images are easily implemented by instanciating a default_image_t object (flat image) and a redolog_t object (redolog).
"growing" disk images only instanciates a redolog_t object.
Classe names are undoable_image_t, volatile_image_t and growing_image_t.
When using these disk images, the underlying data structure and layout is completely hidden to the caller. Then, all offset and size values are "logical" values, as if the disk was a flat file.
Constants
#define UNDOABLE_REDOLOG_EXTENSION ".redolog" #define UNDOABLE_REDOLOG_EXTENSION_LENGTH (strlen(UNDOABLE_REDOLOG_EXTENSION)) #define VOLATILE_REDOLOG_EXTENSION ".XXXXXX" #define VOLATILE_REDOLOG_EXTENSION_LENGTH (strlen(VOLATILE_REDOLOG_EXTENSION)) |
undoable_image_t methods
undoable_image_t(Bit64u size, const char* redolog_name); instanciates a new undoable_image_t object. This disk image logical length is size bytes and the redolog filename is redolog_name.
int open (const char* pathname); opens the flat disk image pathname, as an undoable disk image. The associated redolog will be named pathname with a UNDOABLE_REDOLOG_EXTENSION suffix, unless set in the constructor. Returns 0 for OK or -1 if a problem occured.
void close (); closes the flat image and its redolog.
off_t lseek (off_t offset, int whence); seeks at logical data position offset in the undoable disk image. Only SEEK_SET is supported for whence. Returns -1 if a problem occured, or the current logical offset in the undoable disk image.
ssize_t read (void* buf, size_t count); reads count bytes of data from the undoable disk image, from current logical offset, and copies it into buf. count must be 512. Returns the number of bytes read. Data will be read from the redolog if it has been previously written or from the flat image otherwise.
ssize_t write (const void* buf, size_t count); writes count bytes of data from buf to the undoable disk image, at current logical offset. count must be 512. Returns the number of bytes written. Data will always be written to the redolog.
volatile_image_t methods
volatile_image_t(Bit64u size, const char* redolog_name); instanciates a new volatile_image_t object. This disk image logical length is size bytes and the redolog filename is redolog_name plus a random suffix.
int open (const char* pathname); opens the flat disk image pathname, as a volatile disk image. The associated redolog will be named pathname with a random suffix, unless set in the constructor. Returns 0 for OK or -1 if a problem occured.
void close (); closes the flat image and its redolog. The redolog is deleted/lost after close is called.
off_t lseek (off_t offset, int whence); seeks at logical data position offset in the volatile disk image. Only SEEK_SET is supported for whence. Returns -1 if a problem occured, or the current logical offset in the volatile disk image.
ssize_t read (void* buf, size_t count); reads count bytes of data from the volatile disk image, from current logical offset, and copies it into buf. count must be 512. Returns the number of bytes read. Data will be read from the redolog if it has been previously written or from the flat image otherwise.
ssize_t write (const void* buf, size_t count); writes count bytes of data from buf to the volatile disk image, at current logical offset. count must be 512. Returns the number of bytes written. Data will always be written to the redolog.
growing_image_t methods
growing_image_t(Bit64u size); instanciates a new growing_image_t object. This disk image logical length is size bytes.
int open (const char* pathname); opens the growing disk image pathname, Returns 0 for OK or -1 if a problem occured.
void close (); closes the growing disk image.
off_t lseek (off_t offset, int whence); seeks at logical data position offset in the growable disk image. Only SEEK_SET is supported for whence. Returns -1 if a problem occured, or the current logical offset in the grwoing image.
ssize_t read (void* buf, size_t count); reads count bytes of data from the growing disk image, from current logical offset, and copies it into buf. count must be 512. Returns the number of bytes read. The buffer will be filled with null bytes if data has not been previously written to the growing image.
ssize_t write (const void* buf, size_t count); writes count bytes of data from buf to the growing disk image, at current logical offset. count must be 512. Returns the number of bytes written.